Papers with model families
Detecting Hallucinations in Large Language Models via Internal Attention Divergence Signals (2026.acl-srw)
Copied to clipboard
| Challenge: | Existing methods conflate fluency with correctness or require substantial computational overhead. |
| Approach: | They propose a single-pass uncertainty quantification method that uses attention matrices to estimate uncertainty without requiring repeated sampling or external models. |
| Outcome: | The proposed method performs well across multiple datasets, task types, and model families and is highly predictive of answer correctness. |
A Named Entity Recognition Shootout for German (P18-2)
Copied to clipboard
| Challenge: | Named entity recognition and classification (NER) is a central component in many natural language processing pipelines. |
| Approach: | They propose to build a model for German named entity recognition that performs at the state of the art for both contemporary and historical texts. |
| Outcome: | The proposed model outperforms the CRF and BiLSTM on large and small datasets. |
Evaluating the Impact of SAE-based Language Steering on LLM Performance (2026.eacl-srw)
Copied to clipboard
| Challenge: | Recent advances in Sparse Autoencoders (SAEs) have revealed interpretable features within large language models (LLMs) however, the impact of SAE-based language steering on output quality and task performance remains unclear. |
| Approach: | They apply language-specific SAE feature steering to three LLMs from two model families and evaluate it on a translation task and a multilingual question-answering task. |
| Outcome: | The proposed approach outperforms prompting and language neuron-based steering on translation and multilingual question-answering tasks. |
Aligning What LLMs Do and Say: Towards Self-Consistent Explanations (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) are often prompted to produce natural language explanations, but the features driving the answer are often different from those emphasized in their explanations. |
| Approach: | They propose a large-scale benchmark linking model decisions with diverse explanations and attribution vectors across datasets, methods, and model families to address this gap. |
| Outcome: | The proposed model generates an answer where the word NLP in the prompt has high feature importance. |
Beyond Bias Scores: Unmasking Vacuous Neutrality in Small Language Models (2026.eacl-srw)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are expensive to deploy locally and can reproduce harmful social biases in high-stakes settings such as healthcare and education. |
| Approach: | They propose a multi-dimensional evaluation paradigm to assess SLM fairness prior to deployment. |
| Outcome: | The proposed framework examines model robustness across four stages - biases, utility, ambiguity handling, and positional bias over diverse social bias categories. |
Unsupervised Detection of LLM-Generated Text in Korean Using Syntactic and Semantic Cues (2026.findings-eacl)
Copied to clipboard
| Challenge: | Prior work focused on English, leaving low-resource languages such as Korean underexplored. |
| Approach: | They propose an unsupervised framework that integrates syntactic token cohesiveness and semantic regeneration similarity to detect Korean text. |
| Outcome: | The proposed framework outperforms baselines in Korean and other low-resource languages without training. |
Learning Auxiliary Tasks Improves Reference-Free Hallucination Detection in Open-Domain Long-Form Generation (2025.acl-short)
Copied to clipboard
Chengwei Qin, Wenxuan Zhou, Karthik Abinav Sankararaman, Nanshu Wang, Tengyu Xu, Alexander Radovic, Eryk Helenowski, Arya Talebzadeh, Aditya Tayade, Sinong Wang, Shafiq Joty, Han Fang, Hao Ma
| Challenge: | Existing methods for detecting hallucination in long-form tasks focus on limited domains or rely heavily on external fact-checking tools, which may not always be available. |
| Approach: | They propose a new paradigm that augments fine-tuning with an auxiliary task for the model to jointly learn with the main task of hallucination detection. |
| Outcome: | The proposed method outperforms existing methods for detecting hallucination in open-domain long-form generation and is more accurate than random guessing. |
Are LLMs reliable? An exploration of the reliability of large language models in clinical note generation (2025.acl-industry)
Copied to clipboard
Kristine Ann M. Carandang, Jasper Meynard Arana, Ethan Robert Casin, Christopher Monterola, Daniel Stanley Tan, Jesus Felix B. Valenzuela, Christian Alis
| Challenge: | Clinical note generation (CNG) tools are being developed to address extended working hours and healthcare provider fatigue. |
| Approach: | They evaluate the reliability of 12 open-weight and proprietary LLMs from Anthropic, Meta, Mistral, and OpenAI in CNG in terms of their ability to generate notes that are string equivalent (consistency rate), have the same meaning (semantic consistency) and are correct (symbol similarity) |
| Outcome: | The results show that the LLMs generated notes that are string equivalent (consistency rate), have the same meaning (semantic consistency) and are correct (symbol similarity) overall, Meta’s Llama 70B was the most reliable, followed by Mistral’s Small model. |
Beyond Divergent Creativity: A Human-Based Evaluation of Creativity in Large Language Models (2026.findings-eacl)
Copied to clipboard
| Challenge: | Large language models are increasingly used in verbal creative tasks. |
| Approach: | They propose a divergent association task that focuses on novelty, ignoring appropriateness, a core component of creativity. |
| Outcome: | The proposed model scores are lower than baselines with no creative abilities, undermining its validity for model evaluation. |
Analyzing LLM Instruction Optimization for Tabular Fact Verification (2026.findings-eacl)
Copied to clipboard
Xiaotang Du, Giwon Hong, Wai-Chung Kwan, Rohit Saxena, Ivan Titov, Pasquale Minervini, Emily Allaway
| Challenge: | evaluating instruction optimization for tabular fact verification is a key challenge for reliable NLP systems. |
| Approach: | They compare instruction optimization for tabular fact verification with a framework based on DSPy . they find that instruction optimization consistently improves verification accuracy . |
| Outcome: | The proposed method improves verification accuracy across four benchmarks and three model families. |
Neural Breadcrumbs: Membership Inference Attacks on LLMs Through Hidden State and Attention Pattern Analysis (2026.eacl-long)
Copied to clipboard
| Challenge: | Membership inference attacks (MIAs) reveal whether specific data was used to train machine learning models, serving as important tools for privacy auditing and compliance assessment. |
| Approach: | They propose to examine LLMs’ internal representations rather than just their outputs to gain additional insights into potential membership inference signals. |
| Outcome: | The proposed framework yields strong membership detection across several model families achieving average AUC scores of 0.85 on popular MIA benchmarks. |
The Curious Case of Absolute Position Embeddings (2022.findings-emnlp)
Copied to clipboard
| Challenge: | In natural language, it is not absolute position that matters, but relative position . et al., 2017) language models incorporate positional encodings that encode absolute (linear) word order. |
| Approach: | They find that Transformer language models encode word order using positional information . they also find that models that use absolute position embeddings over-rely on positional data . |
| Outcome: | The results raise questions about the efficacy of APEs to model the relativity of position information. |
Enhancing Cross-Tokenizer Knowledge Distillation with Contextual Dynamical Mapping (2025.findings-acl)
Copied to clipboard
| Challenge: | Knowledge distillation (KD) approaches focus on homogeneous architectures with identical tokenizers, constraining their applicability in cross-architecture scenarios. |
| Approach: | They propose a framework that uses contextual information to enhance sequence alignment precision and dynamically improves vocabulary mapping. |
| Outcome: | The proposed framework shows significant advantages over existing methods for model compression . it can be used across multiple model families and across multiple benchmarks . |
Beyond Positive Scaling: How Negation Impacts Scaling Trends of Language Models (2023.findings-acl)
Copied to clipboard
Yuhui Zhang, Michihiro Yasunaga, Zhengping Zhou, Jeff Z. HaoChen, James Zou, Percy Liang, Serena Yeung
| Challenge: | Recent studies show that some tasks exhibit inverse scaling, or U-shaped scaling, where the performance degrades as models are scaled up. |
| Approach: | They propose a task that asks questions with negation to show positive scaling . they hypothesize that solving NeQA depends on question answering and negation understanding . |
| Outcome: | The proposed task can exhibit inverse scaling, U-shaped scaling, or positive scaling, and the scaling trends shift as the task is more powerful. |
When Debiasing Backfires: Counterintuitive Side Effects of Preprocessing-Based Stereotype Mitigation (2026.findings-acl)
Copied to clipboard
| Challenge: | Preprocessing-based methods for stereotype mitigation are widely used in NLP . preprocessing methods cause unintended shifts in attention flow, authors say . |
| Approach: | They propose to use preprocessing-based methods to reduce stereotypes for targeted groups . they find that stereotyping or counter-stereotyping can increase for other demographics . |
| Outcome: | The proposed methods often induce unintended shifts across demographics, the authors show . they show that such side effects are not accompanied by large changes in attention flow . |
Improving Instruct Models for Free: A Study on Partial Adaptation (2025.emnlp-main)
Copied to clipboard
Ozan Irsoy, Pengxiang Cheng, Jennifer L Chen, Daniel Preotiuc-Pietro, Shiyue Zhang, Duccio Pappadopulo
| Challenge: | Instruct models are deemed superior and more usable but can be eroded by instruction tuning . a recent study shows that instruct models are better at following instructions than base models . |
| Approach: | They scale down the strength of instruction tuning to improve model performance . they show that reducing instruction tuning results in material improvement . |
| Outcome: | The proposed model improves on a few-shot in-context learning benchmark . but it loses some degree of its in-training ability . |
DOVE: A Large-Scale Multi-Dimensional Predictions Dataset Towards Meaningful LLM Evaluation (2025.findings-acl)
Copied to clipboard
Eliya Habba, Ofir Arviv, Itay Itzhak, Yotam Perlitz, Elron Bandel, Leshem Choshen, Michal Shmueli-Scheuer, Gabriel Stanovsky
| Challenge: | Recent work found that LLMs are sensitive to arbitrary prompt dimensions . this challenges traditional single-prompt evaluation practices . |
| Approach: | They present a large-scale dataset containing prompt perturbations of various evaluation benchmarks . they examine LLM sensitivity from an holistic perspective and assess the joint effects of perturbations along various dimensions . |
| Outcome: | The proposed dataset aims to democratize evaluation research and enable robust protocols . it includes more than 250M prompt perturbations and model outputs . |
Nudging: Inference-time Alignment of LLMs via Guided Decoding (2025.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) require alignment to effectively and safely follow user instructions. |
| Approach: | They propose a simple, training-free algorithm that aligns any base model at inference time using a small aligned model. |
| Outcome: | The proposed algorithm outperforms large aligned models on open-instruction tasks without training. |
From Text to Source: Results in Detecting Large Language Model-Generated Content (2024.lrec-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) generate human-like text, but have ethical and misuse concerns. |
| Approach: | They evaluate whether a classifier trained to distinguish between source and target LLMs can detect text from an LLM without further training. |
| Outcome: | The proposed method detects text from target LLMs without further training. |
Evaluating Robustness of Large Language Models Against Multilingual Typographical Errors (2026.acl-long)
Copied to clipboard
| Challenge: | Large language models (LLMs) are increasingly deployed in multilingual, real-world applications where user inputs introduce typographical errors. |
| Approach: | They propose a multilingual typo generation algorithm that simulates human-like errors based on language-specific keyboard layouts and typing behavior. |
| Outcome: | The proposed model can generate the correct answer ("500") under typos in English, German, and Russian. |
Lightweight Haar Wavelet Subband Pruning for LLMs (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have impressive performance but require computational and memory resources. |
| Approach: | They propose a post-training framework that uses a Haar wavelet transform to prune weights. |
| Outcome: | The proposed pruning framework reduces pruning time and computational costs by removing less important weights while preserving model architecture. |
CLewR: Curriculum Learning with Restarts for Machine Translation Preference Learning (2026.findings-acl)
Copied to clipboard
| Challenge: | Large language models (LLMs) have demonstrated competitive performance in zero-shot multilingual machine translation (MT). |
| Approach: | They propose a curriculum learning strategy with restarts which reiterates easy-to-hard curriculum multiple times during training to effectively mitigate catastrophic forgetting of easy examples. |
| Outcome: | The proposed model replicates easy-to-hard curriculum multiple times during training to mitigate catastrophic forgetting of easy examples. |
MotiveBench: How Far Are We From Human-Like Motivational Reasoning in Large Language Models? (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing benchmarks for large language models lack information asymmetry with real-world situations. |
| Approach: | They propose a benchmark to evaluate the human-like motivational and behavioral reasoning ability of LLMs with detailed, realistic situations. |
| Outcome: | The proposed benchmark compared LLMs with real-world scenarios on seven model families and found that the most advanced models struggle with understanding "love & belonging" needs. |
Pattern Enhanced Multi-Turn Jailbreaking: Exploiting Structural Vulnerabilities in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing multi-turn methods for large language models exploit conversational context to bypass safety constraints gradually. |
| Approach: | They propose a framework of five conversation patterns to construct multi-turn jailbreaks through natural dialogue. |
| Outcome: | The proposed framework exploits conversational contexts to construct multi-turn jailbreaks . it reveals that models exhibit distinct weakness profiles and model families share similar failure modes . |
Around the World in 24 Hours: Probing LLM Knowledge of Time and Place (2025.acl-long)
Copied to clipboard
| Challenge: | Existing studies have tested language models' ability to reason over time and space in isolation or only in simple or artificial environments. |
| Approach: | They present a dataset of 320k prompts covering 289 cities in 217 countries and 37 time zones to evaluate their ability to jointly reason over time and space. |
| Outcome: | The proposed models perform well on reasoning tasks involving only temporal knowledge, but performance remains constrained on tasks that require connecting temporal and geographic information. |
When Efficiency Meets Safety: A Benchmark Security Analysis of KV Cache Compression in Large Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Key-Value (KV) caching is widely used in large language models to enable long-context inference efficiently, yet its security implications remain underexplored. |
| Approach: | They propose a history-aware, per-head feedback merging strategy that prevents safety degradation while maintaining efficiency. |
| Outcome: | The proposed strategy prevents safety degradation while maintaining efficiency. |
The Role of Outgoing Connection Heterogeneity in Feedforward Layers of Large Language Models (2025.emnlp-main)
Copied to clipboard
| Challenge: | Using a new fine-tuning loss, we show that inner neurons with diverse outgoing connections are more critical to model performance than those with uniform connections. |
| Approach: | They propose a new loss that reduces the outgoing connection entropy in feedforward layers and elucidates the role of outgoing connections in large language models. |
| Outcome: | The proposed method is significantly more effective than removing neurons randomly or based on their magnitude. |
Stop When Enough: Adaptive Early-Stopping for Chain-of-Thought Reasoning (2026.acl-long)
Copied to clipboard
| Challenge: | Chain-of-Thought reasoning has driven recent gains of large language models (LLMs) on reasoning-intensive tasks by externalizing intermediate steps. |
| Approach: | They propose a training-free framework that adaptively determines when to stop reasoning to mitigate overthinking. |
| Outcome: | The proposed framework reduces token usage by 20-55% while maintaining or improving accuracy compared to standard CoT prompting. |
ManCC: A Task-Anchored Benchmark for Manchu–Classical Chinese Cross-Lingual Modeling (2026.findings-acl)
Copied to clipboard
Meiqi Wang, Xiaoxin Sun, Dongjie Wang, Ruixin Yu, Xiantao Heng, Shuo Wang, Zhen Huang, Peng Zhao, Suhua Wang, Minghao Yin
| Challenge: | Mainstream research in natural language processing has focused on high-resource and modern languages. |
| Approach: | They propose a task-anchored benchmark for Manchu–Classical Chinese translation . they use a parallel corpus of 16,627 sentence pairs to evaluate the model . |
| Outcome: | The proposed benchmarks show that linguistic differences influence performance and broader language coverage facilitate low-resource transfer. |
Do Personality Traits Interfere? Geometric Limitations of Steering in Large Language Models (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing studies on personality steering in large language models rely on injecting trait-specific steering vectors into the residual stream to control the strength of trait expression. |
| Approach: | They examine the geometric relationships between Big Five personality steering directions by applying geometric conditioning schemes to their steering vectors. |
| Outcome: | The proposed model can be used to steer personality traits in large language models. |
Machine-generated text detection prevents language model collapse (2025.emnlp-main)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are increasingly prevalent across the web, resulting in a degenerative process whereby LLMs reinforce their own errors and reduce output diversity. |
| Approach: | They propose to use machine-generated text to reduce model collapse by up-sampling likely human content in training data. |
| Outcome: | The proposed approach prevents model collapse and improves performance compared to training on purely human data. |
Analyzing and Modeling LLM Response Lengths with Extreme Value Theory: Anchoring Effects and Hybrid Distributions (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches treat length as an incidental output property rather than a statistically regular phenomenon worthy of rigorous modeling. |
| Approach: | They propose a statistical framework for modeling and controlling large language model response lengths using extreme value theory and cross-validation on Qwen and DeepSeek architectures. |
| Outcome: | The proposed model improves tail fit and generalizability while maintaining generalizzability. |
Fast and Accurate Fisher-Guided Quantization via Efficient Kronecker Factorization (2026.acl-long)
Copied to clipboard
| Challenge: | Quantization has shown strong results in preserving model quality under compression, but under aggressive bit-width reductions, even quantization may require additional information to prevent performance degradation. |
| Approach: | They propose a Kronecker-factored approximation that captures second-order curvature information, captured by the Hessian, to achieve a 10 speedup over prior approaches. |
| Outcome: | The proposed method significantly accelerates the most expensive component in second-order quantization – Hessian parameterization . it achieves up to a 10 speedup over prior approaches. |
Cell-Based Representation of Relational Binding in Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Recent work has found evidence that Large Language Models (LLMs) are able to track entities across discourse . however, the mechanism by which they bind entities, relations, and attributes remains unclear . |
| Approach: | They propose a low-dimensional cell-based binding representation for relational binding . they also show that context-specific CBR representations are related by translation vectors . |
| Outcome: | The proposed model encodes a low-dimensional cell-based binding representation (CBR) a translation vector in activation space enables cross-context transfer, the study shows . |